'^■'^•^■^-'-TiiTBlllltiitrinlllilAllfMliililtrill 


The  Measurement  of 
Customer  Satisfaction 


FBMS 
1985 
C.  1 


INPUT 


About  INPUT 


INPUT  provides  planning  information, analysis, and 
recomnnendations  to  managers  and  executives  in  the 
information  processing  industries.  Through  market 
research,  technology  forecasting,  and  competitive 
analysis,  INPUT  supports  client  management  in 
making  informed  decisions.  Continuing  services  are 
provided  to  users  and  vendors  of  computers, 
communications,  and  office  products  and  services. 

The  company  carries  out  continuous  and  in-depth 
research.  Working  closely  with  clients  on  important 
issues,  INPUT'S  staff  members  analyze  and  inter- 
pret the  research  data,  then  develop  recommen- 
dations and  innovative  ideas  to  meet  clients'  needs. 


Clients  receive  reports,  presentations,  access  to  data 
on  which  analyses  are  based,  and  continuous 
consulting. 

Many  of  INPUT'S  professional  staff  members  have 
nearly  20  years'  experience  in  their  areasof  speciali- 
zation. Most  have  held  senior  management  positions 
in  operations,  marketing,  or  planning.  This  exper- 
tise enables  INPUT  to  supply  practical  solutions 
to  complex  business  problems. 

Formed  in  1974,  INPUT  has  become  a  leading 
international  planning  services  firm.  Clients  include 
over  100  of  the  world's  largest  and  most  techni- 
cally advanced  companies. 


Offices 


NORTH  AMERICA 


EUROPE 


ASIA 


Headquarters 

1943  Landings  Drive 
Mountain  View,  CA 
94043 

(415)  960-3990 
Telex  171407 

Detroit 

220  East  Huron 
Suite  209 

Ann  Arbor,  Ml  48104 
(313)  971-0667 

New  York 

Park  80  Plaza  West-1 
Saddle  Brook,  NJ  07662 
(201)  368-9471 
Telex  134630 

Washington,  D.C. 

1 1820  Parklawn  Drive 

Suite  201 

Rockville,  MD  20852 
(301)  231-7350 


United  Kingdom 

INPUT,  Ltd. 
41  Dover  Street 
London  W1X  3RB 
England 
01-493-9335 
Telex  27113 

France 

La  Nacelle 

Procedure  d'abonnement  1-74 

2,  rue  Campagne  Premiere 

75014  Paris 

France 

322.56.46 

Telex  220064  X5533 
Italy 

Nomos  Sistema  SR  L 
20127  Milano 
Via  Soperga  36 
Italy 

Milan  284-2850 
Telex  321137 


Japan 

ODS  Corporation 

Dai-ni  Kuyo  BIdg. 

5-10-2,  Minami-Aoyama 

Minato-ku, 

Tokyo  107 

Japan 

(03)  400-7090 
Telex  26487 

Singapore 

Cyberware  Consultants  (PTE)  Ltd. 

2902  Pangkor 

Ardmore  Park 

Singapore  1025 

734-8142 


Sweden 

Athena  Konsult  AB 

Box  22232 

S-104  22  Stockholm 

Sweden 

08-542025 

Telex  17041 


INPUT 

Planning  Services  For  Management 


THE   MEASUREMENT  OF 
CUSTOMER  SATISFACTION 


THE  MEASUREMENT  OF  CUSTOMER  SATISFACTION 


ABSTRACT 


As  customer  service  becomes  increasingly  market  oriented,  customer  satisfaction 
surveys  are  becoming  even  more  important.  Previous  INPUT  studies  have  shown  that 
the  long-term  route  to  profitable  customer  services  is  through  improving 
satisfaction. 

INPUT'S  latest  European  report,  The  Measurement  of  Customer  Satisfaction, 
examines  the  benefits  to  be  gained  from  customer  satisfaction  surveys  and  shows 
how  the  results  of  such  surveys  can  be  analysed  and  data  interpreted  to  focus 
attention  on  the  real  problem. 

This  report  contains  47  pages,  including  32  exhibits.  " 
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INTRODUCTION 


The  purpose  of  this  report,  part  of  INPUT'S  Customer  Service  Program,  is  to 
examine  the  benefits  to  be  gained  from  effective  measurement  of  customer 
satisfaction.  It  explores  the  concepts  and  techniques  of  measurement  and 
shows  examples  of  customer  satisfaction  charts  derived  from  a  number  of 
INPUT  surveys. 


The  report  draws  on  INPUT'S  many  years  of  experience  in  measuring  customer 
satisfaction  and  interpreting  the  results,  and  has  drawn  from  surveys 
conducted  in  1984  and  1985.  The  data  used,  although  "live,"  is  intended  to  be 
indicative  and  does  not  therefore  cover  each  individual  market. 


A  distinction  is  also  drawn  between  what  users  describe  as  their  ideal  wants 
and  their  real  need,  which  is,  in  many  cases,  a  lower  level  of  service  than 
their  ideal  and  a  level  which  can  be  more  profitably  attained. 
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II        THE  IMPORTANCE  OF  MEASUREMENT 


A.       WHY  MEASURE  SATISFACTION? 


•  The  current  issues  in  customer  service,  particularly  due  to  its  increasingly 
competitive  nature,  are  breaking  down  traditional  'brand  loyalties,'  with  the 
result  that  users  are  increasingly  ready  to  satisfy  their  service  needs  from  a 
wider  range  of  suppliers.  As  is  said  in  Exhibit  II- 1,  maintenance  is  now 
becoming  a  commodity. 

•  Price,  although  often  quoted  by  users  as  the  primary  reason  for  changing 
suppliers,  is  not  the  most  important  criterion.  Users  put  service  quality  first, 
with  price  only  becoming  an  issue  once  the  required  level  of  service  has  been 
achieved.  It  is  vital  that  service  vendors  know  precisely  what  is  expected  of 
them  by  their  customers. 

•  Maintenance  is  a  negative  activity  in  the  eyes  of  the  user.  Service,  on  the 
other  hand,  can  be  marketed  as  a  positive  activity.  In  both  cases,  it  is 
important  to  establish  the  criteria  against  which  performance  can— and  will- 
be  measured.  In  this  way  vendors  can  identify  problems  before  they  become 
crises,  and  users  will  have  clear  facts  on  which  to  base  their  discussions  with 
vendors. 

•  Improving  service  quality  can  be  an  end  it  itself,  but  more  significantly,  it  can 
be  a  way  to  improving  profits.  Measuring  the  service  provided  is  an  important 
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EXHIBIT  II-l 


MAINTENANCE  BECOMING  A  COMMODITY 


•      Sharp  Increases  in  Reliability  Encourage  Users  to  Think  of 
Maintenance  as  a  Declining  Need 


•      Service  as  "Commodity"  Means: 


Brand  Name  Loyalty  Decreases 


Service  Market  Opens  to  Competition  which  in  Turn  Causes.  .  . 


Pressure  on  the  Price  of  Maintenance 


•      Equipment  Manufacturers/Service  Vendors  Must: 


Distance  Themselves  from  "Maintenance  Only"  Image 


Develop  Image  of  Total  Service  Company 


Integrate  all  Post-Sale  Services 
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aspect  of  that  process,  as  without  measurement,  it  is  difficult  to  ensure  the 
right  product  is  being  delivered  (see  Exhibit  11-2). 

•  With  user  views  about  service  pricing  hardening— that  is,  with  users  feeling 
increasingly  that  they  are  not  receiving  good  value  for  money— providing  a 
measured  service  is  one  way  in  which  users  can  be  made  aware  of  the  quality 
of  service  that  is  being  provided.  Exhibit  11-3  highlights  the  difference 
between  user  views  of  value  compared  to  vendor  views.  The  gap  between  user 
and  vendor  views  has  serious  implications: 

Users  may  be  tempted  to  buy  apparently  cheaper  service  alternatives. 

User  expectations  may  be  higher  than  they  should  be. 

Vendor  price  flexibility  is  limited. 

B.       FACTORS  TO  MEASURE 

I.  INTRODUCTION 

•  The  two  key  elements  in  measuring  the  effectiveness  of  customer  service  are 
(I)  using  a  factor  which  can  be  measured  in  a  mutually  agreeable  way,  and  (2) 
being  sure  this  factor  is  important  to  the  user, 

•  A  previous  INPUT  study,  User  Requirements  in  Customer  Service,  showed  the 
key  service  elements  to  be: 

System  reliability. 

System  availability. 

Response  time. 
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EXHIBIT  11-2 
WHY  MEASURE  SATISFACTION? 


Effective  CS  Measurement  Ensures 
Customer  Satisfaction 
Optimum  Resource  Utilisation 


Resulting  in: 

-  Profitable  Revenue  Growth 
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EXHIBIT  II-3 

SERVICE  VALUE  -  USER  AND  VENDOR  VIEWS  COMPARED 


Systems 


Peripherals  and 
Terminals 


Office  Automation 
Equipment 


Copiers 


1 


J  L 


Rating 

1  =  Low  Value  for  Money,   10  =  High  Value  for  Money 


V 


A 


Vendors'  Views 
Users'  Views 
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Each  of  these  factors  can  be  measured  at  a  global  level,  but  there  are  a 
number  of  sub-factors  which  must  also  be  taken  into  account  if  vendors  are  to 
optimise  their  support  capability. 

Service  parts  availability  ratio. 

Effectiveness  of  preventive  maintenance. 

Fixing  faults  at  first  call. 

RELIABILITY 

System  reliability  can  be  measured  in  various  ways: 

Mean  time  between  failures. 

System  availability. 

Users  surveyed  by  INPUT  had,  in  general,  no  clearly  defined  method  of 
measuring  reliability,  but  used  system  availability  as  the  key  yardstick. 
Nevertheless,  users  do  have  an  intuitive  view  of  product  reliability.  As  can  be 
seen  in  Exhibit  11-4,  these  views  correspond  roughly  with  vendor  views.  Only 
in  the  case  of  word  processors  and  copiers  is  there  any  significant  divergence 
in  view. 

SYSTEM  AVAILABILITY 

Reliability,  as  discussed  previously,  is  based  essentially  on  perception  rather 
than  any  scientific  measurement.  System  availability,  therefore,  becomes  a 
more  critical  factor  since  it  is  measured  by  both  users  and  vendors. 

Having  said  that,  several  users  INPUT  telephoned  admitted  that  their 
measurement  is  not  always  accurate  and  that  frequently  they  rely  on 
the  vendors'  records. 
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EXHIBIT 


USER  AND  VENDOR  VIEWS  OF  PRODUCT  RELIABILITY 


Large  Systems 


Small  Systems 


Peripherals  S  Terminals 


Data  Communications 


Personal  Computers 


Vi/ord  Processors 


Work  Stations 


Copiers 


USER  VIEW  POORER 
THAN  VENDOR 


USER  VIEW  BETTER 
THAN  VENDOR 


2.4 


0.6 


0.  5 


1 


0.  1 


0,  7 


1 


012 


1.4 


1 


2.5      2.0       1.5     1.0      0.5      0       0.5      1,0      1.5      2.0  2.5 


Points  differentiate  between  users'  rating  of  reliability 
and  vendors'  rating  of  reliability 
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Users  have  two  levels  of  need.  At  the  highest  level  is  their  ideal  target- 
availability,  but  there  is  a  second,  lower  level— the  minimum  acceptable  level 
of  availability.  A  comparison  of  the  1984  results  is  shown  in  Exhibit  11-5. 

In  almost  every  case,  vendors  are  failing  to  meet  the  users'  ideal 
standard,  but  are  exceeding  the  minimum  acceptable  performance. 

Vendors  are  therefore  faced  with  two  options.  First,  they  can  continue  to 
improve  system  availability,  involving  as  it  does  improving  inherent  product 
reliability,  response  time,  and  repair  turnaround  time  for  both  hardware  and 
software  products.  Steadily  improving  product  reliability,  which  in  turn 
improves  system  availability,  is  also  negatively  impacting  user  views  about 
service  value.  The  alternative  is  to  educate  users  to  think  in  terms  of  their 
real  availability  need  rather  than  the  more  nebulous  ideal  target  currently 
considered. 

The  importance  of  system  availability  is  shown  in  Exhibit  11-6,  with  over  half 
the  users  surveyed  rating  system  availability  as  the  most  important  factor  in 
customer  support. 

Exhibit  11-7  demonstrates  that  user  system  availability  requirements  are,  at 
the  ideal  level,  more  demanding  than  vendors  are  currently  able  to  deliver. 
By  comparison,  current  availability  levels  are  above  the  minimum  acceptable 
to  users.  The  two  exceptions  to  this  are: 

Data  communications  equipment,  where  vendor  performance  already 
meets  the  user  ideal. 

Copiers,  where  current  availability  falls  below  the  user  minimum 
acceptable  level. 
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EXHIBIT  II-5 


SYSTEM  AVAILABILITY  -  ACTUAL  COMPARED  TO  IDEAL 
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EXHIBIT  II-6 


SYSTEM  AVAILABILITY  -  USER  RANKING  OF  IMPORTANCE 


PERCENT  OF  USERS  RANKING 
RESPONSE  TIME  AS  : 

TOTAL  PER- 
CENT OF  USERS 

Most 
Important 
Factor 

Important 
Factor 

Neutral 

Relatively 

Unimportant 

Least 
Important 

Important 

Unimportant 

Large  Systems 

71.  3% 

28.7% 

100.0% 

NIL 

Small  Systems 

58.  3 

36.2 

2.5 

2.5 

0.  5 

94.  5 

3.0 

Peripherals  and  Terminals 

46.2 

45.8 

3.0 

4.7 

0.  3 

92.0 

5.0 

Data  Communications 

IT  A  C 

50.  D 

h7 ,  0 

n  Q 
U.  o 

1  c 

1  .  b 

Personal  Computers 

52.6 

38.7 

5.  1 

3.6 

91.3 

3.6 

Word  Processors 

57.6 

39.  4 

1.5 

1.5 

97.0 

1.5 

Work  Station 

65.0 

30.0 

5.0 

95.0 

NIL 

Copiers 

57.  4 

37.  0 

5.6 

94.4 

5.6 

PBX 

82.9 

17.  1 

100.0 

NIL 

Total  Hardware 

56.2 

38.4 

2.  3 

2.  9 

0.2 

94.6 

3.  1 

Source:  INPUT  Survey 
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Examples  of  the  way  in  which  comparative  data  can  be  used  are  shown  in 
Exhibits  11-8  through  11-15.  Exhibit  11-8  compares  the  success  rates  of  a 
number  of  vendors  in  providing  the  desired  service  level,  while  Exhibit  11-9 
shows  the  proportion  of  users  who  are  dissatisfied  with  the  level  of  service. 
The  key  benefit  of  such  analysis  is  that  it  identifies  areas  where  effort  can 
provide  real  improvement  and  can  help  to  eliminate  effort  which  will  satisfy 
few,  if  any,  additional  customers. 

Exhibits  11-16  through  11-26  show  similar  analyses  for  response  times. 
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EXHIBIT  II-8 
LARGE  SYSTEMS  AVAILABILITY  -  U.K. 


Number 
of 

EQUIPMENT  AVAILABILITY 
(Percent) 

PERCENT  OF 
USERS  RECEIVE 

MANUFACTURER 

Observa- 
tions 

Current 

Ideal 

Minimum 
Acceptable 

Ideal 

Minimum 
Acceptable 

ICL 

27 

95.3% 

96.  5% 

92.  9% 

51.9% 

88 . 9% 

IBM 

27 

97.0 

96.8 

93.6 

55.5 

96.  3 

DEC 

7 

96.6 

95.3 

88.9 

57.  1 

85.7 

Burroughs 

7 

96.  U 

98.0 

93.9 

42.9 

85.  7 

Honeywell 

n 

96.8 

96.  3 

93.4 

36.4 

100.0 

Sperry 

3 

99.6 

96.7 

96.7 

100.0 

100.0 

Hewlett-Packard 

97.8 

98.3 

96.7 

50.0 

75.0 

Other 

12 

96.8 

97.6 

94.  2 

50.0 

75.0 

Total 

98 

96.5% 

96.9% 

93 .3% 

52.6% 

91.  8% 

Source:  INPUT  Survey 
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EXHIBIT  II-9 


U.K.  LARGE  SYSTEMS 
USER  SAT8SFACTI0N  WITH  AVAILABILITY 


100.0% 


100% 


90 


50 


40 


30 


20 


87.6% 


56.7% 


User  Minimum  Acceptable 
—  93 . 3% 


10.3%    10.3%    10.3%  10.3% 


40.2 


37.  1% 


12.4% 


<90%         90  91 

Dissatisfied  Users 


92  93         94  95  96 

System  Availability  (Percent) 
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EXHIBIT  11-10 


PERIPHERAL  AND  TERMINAL  AVAILABILITY  -  U.K. 


Number 

OT 

EQUIPMENT  AVAILABILITY  1 
(Percent) 

PERCENT  OF 
USERS  RECEIVE 

MANUFACTURER 

Observa- 
tions 

Current 

Ideal 

Minimum 
Acceptable 

Ideal 

Minimum 
Acceptable 

IBM 

49 

98.  0% 

97.  4% 

96.  9% 

67.  3% 

95.  6% 

ICL 

47 

91.  5 

94.6 

89.8 

53.2 

78.7 

Honeywell 

18 

93.8 

91.6 

89.2 

61.  1 

83.3 

Memorex 

15 

92.  3 

91.  3 

89.  4 

73.3 

93.  3 

DEC 

13 

96.  1 

91.5 

86.  7 

92.  3 

100.  0 

Hewlett-Packard 

9 

96.  8 

95.  7 

94.  7 

88.  9 

88.  9 

Burroughs 

8 

95.8 

96.  1 

93.9 

75.0 

100.0 

Systime 

8 

94.  1 

94.  1 

92.2 

100.0 

100.0 

STC 

8 

90.  1 

89.9 

88.  4 

75.0 

87.  5 

NCR 

6 

90.5 

91.  8 

86.7 

66.7 

100.  0 

Philips 

5 

88.6 

89.0 

84.6 

80.0 

80.0 

Other 

53 

84.6 

83.7 

79.  4 

64.2 

90.6 

Total 

239 

92.  1% 

91.  9% 

88.9% 

67.8% 

90.0% 

Source:  INPUT  Survey 
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EXHIBIT  11-11 


U.K.  PERIPHERALS  AND  TERMINALS 
USER  SATISFACTION  WITH  AVAILABILITY 


(/I 
Z) 


100% 


90 


80 


70 


60 


100.0% 


c 

<u 

^  50 

Q- 
> 


10 


0 


User  iViinimum  Acceptable  =  88.9% 


50.9? 


45.2% 


46.5% 


—      27.0%    27.0%    27.0%  27.0^ 


29.6% 


93.0% 


66.  1% 


<90%         90  91  92  93         94  95  96 

System  Availability  (Percent) 
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EXHIBIT  11-12 


DATA  COMMUNICATIONS  EQUIPMENT  AVAILABILITY 


Number 
of 

EQUIPMENT  AVAILABILITY 
(Percent) 

PERCENT  OF 
USERS  RECEIVE 

MANUFACTURER 

Observa- 
tions 

Current 

Ideal 

Minimum 
Acceptable 

Ideal 

Minimum 
Acceptable 

Racal 

28 

96.  2% 

97.  4% 

93.  6% 

71  4?; 

Qfi  ll9- 

British  Telecom 

16 

98.7 

98.2 

95.  4 

68.8 

93.8 

Case 

12 

97.  9 

98.8 

97.  5 

83.3 

83.  3 

K  It' 

Micom 

8 

96.7 

96.6 

95.  1 

87.5 

87.5 

IBM 

5 

89.4 

88.0 

87.0 

100.0 

100.0 

Codex 

98.8 

98.  5 

97.  8 

75.0 

75.0 

Thorn  EMI 

4 

99.2 

100.0 

98.0 

25.0 

100.0 

Other 

33 

92.0 

93.4 

92.  3 

63.6 

87.  9 

Total 

110 

95.4% 

96.  1% 

94.0% 

70.  9% 

90.9% 

-19- 

©1985  by  INPUT.  Reproduction  Prohibited.  INPUT 

FBMS 


EXHIBIT  11-13 


DATA  COMMUNICATIONS  EQUIPMENT 
USER  SATISFACTION  WITH  AVAILABILITY 


100. 0^ 


100% 


90 


80 


70 


60 


c 

^  50 
Q. 


J5  40 

E 
E 
3 
U 

30 


20 


10 


66.4% 


Ideal  =  96.  r 


24.5%  24.5% 


26.4 


—     17.3%    17.3%    17.3%  "^7,3% 


<90^ 


90 


91 


Dissatisfied  Users 


92  93         94  95  96 

System  Availability  (Percent) 
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EXHIBIT  11-14 


OFFICE  AUTOMATION  EQUIPMENT  AVAILABILITY 


Number 
of 

EQUIPMENT  AVAILABILITY 
(Percent) 

PERCENT  OF 
USERS  RECEIVE 

MANUFACTURER 

Observa- 
tions 

Current 

Ideal 

Minimum 
Acceptable 

Ideal 

Minimum 
AccqDtable 

IBM 

17 

96.2% 

93.7% 

91.0% 

64.7% 

76.5% 

ICL 

12 

93.0 

96.0 

92.6 

58.3 

83.3 

Apple 

8 

95.5 

90.  8 

89.4 

62.  5 

75.0 

ACT 

7 

99.0 

92.  3 

89.  4 

57.  1 

100.0 

Wang 

7 

97.0 

96.  7 

94.0 

57.  1 

100.0 

Hewlett-Packard 

4 

81.8 

80.  5 

77.  8 

75.0 

75.0 

Philips 

4 

93.2 

92.5 

82.5 

75.0 

75.0 

Other 

40 

88.2 

90.7 

86.6 

62.5 

80.0 

Total 

99 

92.  1% 

92.  1% 

88.  5% 

62.6% 

81.8% 
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EXHIBIT  11-15 


OFFICE  AUTOMATION  EQUIPMENT 
USER  SATISFACTION  WITH  AVAILABILITY 


100.0% 


<90%         90  91  92  93         94  95  96  97  98  99  100?, 


System  Availability  (Percent)  Source:  INPUT  Survey 

Dissatisfied  Users 
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EXHIBIT  11-16 


RESPONSE  TIME  -  USER  RANKING  OF  IMPORTANCE 


PERCENT  OF  USERS  RANKING 
RESPONSE  TIME  AS: 

TOTAL  PER- 
CENT OF  USERS 

Most 
Important 
Factor 

Important 
Factor 

Neutral 

Relatively 

Unimportant 

Least 
Important 

Important 

Unimportant 

Large  Systems 

8.6% 

90.  3% 

1.1% 

98.9% 

1.1% 

Small  Systems 

8.  8 

86.  8 

2.5% 

1.9 

95.  6 

1.9 

Peripherals  and  Terminals 

9.0 

83.  3 

4.3 

2.7 

0.  7% 

92.  3 

3.4 

Data  Communications 

6.2 

88.4 

1.5 

3.  9 

- 

94.  6 

3.  9 

Personal  Computers 

3.6 

85.0 

6.4 

4,  3 

0.  7 

88,  6 

5.0 

Word  Processors 

8.6 

87.  1 

2.9 

1.4 

95.  7 

1.4 

Vt/ork  Stations 

10.0 

75.0 

10.0 

5.0 

85.0 

5.0 

Copiers 

8.  4 

81.4 

1.7 

8.5 

89.  8 

8.  5 

PBX 

16.  6 

77.  8 

5.6 

94.4 

5.  6 

Total  Hardware 

8.0% 

85.  2% 

3.  3% 

3.  1% 

0.  4% 

93.2% 

3 .  5  '3 
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EXHIBIT  11-17 


RESPONSE  TIMES  -   CURRENT  SERVICE  EFFECTIVENESS 

AVERAGES 


Number 
of 

RESPONSE  TIME 

PERCENT  OF 
USERS  RECEIVE 

UDserva 
tions 

Current 

1  deal 

T  .  O  .  P .  * 

ideal 

T"          r>  4. 

T  .  0  .  P .  * 

1— di  ye  oysisms 

O  J 

2.6 

1.8 

4.4 

43.3% 

92.8% 

Small  Systems 

152 

4.  1 

2.7 

5.7 

36.2 

87.  5 

Peripherals  and 
Terminals 

97 

5.  1 

3.  3 

7.  9 

45.  4 

87.6 

Data  Communica- 
tions 

75 

5.7 

2.  5 

6.5 

16.0 

74.7 

Personal  Computers 

69 

9.6 

4.2 

12.6 

27.  5 

73.9 

Word  Processors 

37 

4.  1 

2.6 

6.  1 

29.7 

91.9 

Copiers 

34 

5.0 

3.0 

8.  9 

32.4 

88.2 

♦"Threshold  of  Pain"  =  Longest  delay  acceptable  to  users 
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EXHIBIT  11-18 


RESPONSE  TIMES  -  CURRENT  SERVICE  EFFECTIVENESS 

STANDARD  DEVIATION 


IN  Uiii  Uc  r 

of 

RESPONSE 

Tl 

IVI  E- 

Observa- 
tions 

Current 

1  deal 

T.O.P.* 

Large  Systems 

83 

1.44 

1.40 

3.46 

Small  Systems 

152 

4.70 

2.37 

3.89 

Peripherals  and 
Terminals 

97 

4.43 

3.  98 

8. 18 

Data  Communica- 
tions 

75 

6.01 

1.  91 

6.  85 

Personal  Computers 

69 

9.  39 

6.79 

12.82 

Word  Processors 

37 

2.01 

2.00 

3.88 

Copiers 

34 

3»37 

3.31 

9.  47 

♦'Threshold  of  Pain"  =  Longest  delay  acceptable  to  users. 
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EXHIBIT  11-19 


RESPONSE  TIMES:  CURRENT  SERVICE  EFFECTIVENESS 
LARGE  SYSTEMS  AVERAGES 


Number 
of 

Observa- 

f  inn  ^ 

RESPONSE 

TIME 

PERCENT  OF 
USERS  RECEIVE 

Current 

Ideal 

T.O.P.* 

Ideal 

T.O.P. * 

Honeywell 

7 

2.2 

1.8 

■S 

4.  1 

57.  1 

85.7 

ICL 

12 

2.4 

1.7 

3.8 

41.7% 

91.7% 

DEC 

17 

3.7 

2.8 

7.  3 

35.  3 

100.0 

IBM 

22 

2.4 

1.6 

3.0 

30.4 

78.3 

*"ThreshoId  of  Pain"  =  Longest  delay  acceptable  to  users. 
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EXHIBIT  11-20 


RESPONSE  TIMES:  CURRENT  SERVICE  EFFECTIVENESS 
LARGE  SYSTEMS  STANDARD  DEVIATION 


Number 
of 

Observa- 
tions 

RESPONSE 

Tl 

ME 

Current 

Ideal 

T.O.P.* 

Honeywell 

7 

0.99 

1.06 

2.  70 

ICL 

12 

1.  40 

0.96 

2.05 

DEC 

17 

1.72 

1.98 

5.65 

IBM 

22 

1.03 

0.48 

1.50 

*"Threshold  of  Pain"  =  Longest  delay  acceptable  to  users. 
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RESPONSE  TIMES:  CURRENT  SERVICE  EFFECTIVENESS 
SMALL  SYSTEMS  AVERAGES 


Number 
of 

Observa- 
tions 

RESPONSE 

TIME 

PERCENT  OF 
USERS  RECEIVE 

Current 

Ideal 

T.O.P.* 

Ideal 

T.O.P. * 

DEC 

32 

4.8 

3.2 

5.  9 

53.  1% 

87.5% 

Honeywell 

8 

3.8 

2.  3 

4.  5 

37.5 

75.0 

ICL 

12 

3.  9 

2.  1 

4.  9 

33.3 

75.0 

IBM 

22 

3.  3 

1.9 

6.2 

18.2 

90.9 

Data  General 

12 

U.2 

1.9 

6.0 

16.7 

100.0 

*"Threshold  of  Pain"  =  Longest  delay  acceptable  to  users. 
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EXHIBIT  11-22 


RESPONSE  TIMES:  CURRENT  SERVICE  EFFECTIVENESS 
SMALL  SYSTEMS  STANDARD  DEVIATION 


Number 
of 

Observa- 
tions 

RESPONSE 

Tl 

ME 

Current 

Ideal 

T.O.P.* 

DEC 

32 

6.  70 

2.02 

3.86 

Honeywell 

8 

3.  32 

2.32 

2.  55 

ICL 

12 

2.90 

1.85 

2.  40 

IBM 

22 

1.65 

1.69 

4.81 

Data  General 

12 

2.  30 

1.04 

3.29 

*"Threshold  of  Pain"  =  Longest  delay  acceptable  to  users. 
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EXHIBIT  11-23 


RESPONSE  TIMES:  CURRENT  SERVICE  EFFECTIVENESS 
PERIPHERALS  AND  TERMINALS  AVERAGE 


Number 
of 

Observa- 
tions 

RESPONSE 

TIME 

PERCENT  OF 
USERS  RECEIVE 

Current 

Ideal 

T.O.P.* 

Ideal 

T.O.P. * 

Hewlett-Packard 

8 

9.0 

7.6 

16.  9 

75.0 

100.0 

DEC 

24 

5.6 

4.  5 

10.0 

54.2 

91.7 

Honeywell 

9 

4.4 

1.7 

6.  3 

44.4 

66.7 

IBM 

27 

4.3 

2.  4 

6.  5 

37.0% 

81.4% 

ICL 

10 

3.  8 

1.6 

3.7 

20.0 

70.  0 

*"Threshold  of  Pain"  =  Longest  delay  acceptable  to  users. 
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EXHIBIT  11-24 


RESPONSE  TIMES:  CURRENT  SERVICE  EFFECTIVENESS 
PERIPHERALS  AND  TERMINALS  STANDARD  DEVIATION 


IN  u  III  ue  r 
of 

Observa- 
tions 

RESPONSE  Tl 

MP 
IVI  c 

Current 

Ideal 

Hewlett-Packard 

8 

10.  91 

10.91 

19.  14 

DEC 

24 

3.65 

2.76 

7.  97 

Honeywell 

9 

4.00 

0.25 

1.00 

IBM 

27 

2.77 

1.77 

4.03 

ICL 

10 

2.  14 

0.47 

1.85 

*'Threshold  of  Pain"  =  Longest  delay  acceptable  to  users. 
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EXHIBIT  11-25 


RESPONSE  TIMES:  CURRENT  SERVICE  EFFECTIVENESS 
DATA  COMMUNICATIONS  EQUIPMENT  AVERAGES 


Number 
of 

Observa- 
tions 

RESPONSE 

TIME 

PERCENT  OF 
USERS  RECEIVE 

Current 

Ideal 

T.O.P.* 

Ideal 

T.O.P. * 

British  Telecom 

8 

6.0 

1.8 

4.  5 

25.0 

75.0 

Racal 

15 

2.7 

1.8 

4.3 

20.0% 

86.7% 

CASE 

11 

8.4 

3.0 

7.2 

9.  1 

81.8 

*"Threshold  of  Pain"  =  Longest  delay  acceptable  to  users. 
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EXHIBIT  11-26 


RESPONSE  TIMES:  CURRENT  SERVICE  EFFECTIVENESS 
DATA  COMMUNICATIONS  EQUIPMENT  STANDARD  DEVIATION 


Number 
of 

Observa- 
tions 

RESPONSE 

T 

ME 

Current 

1  deal 

T.O.P.* 

British  Telecom 

8 

6,  87 

1.  00 

2.  24 

Racal 

15 

1 . 00 

0.  98 

2.  27 

CASE 

n 

8.  18 

2.  17 

6.  32 

*"Threshold  of  Pain"  =  Longest  delay  acceptable  to  users. 
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Ill       USER  PERCEPTIONS  OF  SATISFACTORY  SERVICE 


A.       IMPORTANCE  OF  SERVICE  CRITERIA 

•  When  measuring  customer  satisfaction,  it  is  important  to  ensure  that  the 
relative  importance  of  the  several  service  elements  being  considered  is  estab- 
lished. When  putting  together  the  final  action  plan,  it  will  then  be  possible  to 
focus  attention  on  the  most  important  areas,  and  scarce  resources  are  not 
wasted  on  items  which  will  not  materially  improve  the  users'  satisfaction. 

•  Two  main  approaches  can  be  adopted. 

Users  can  be  asked  to  rank  various  elements  in  order  of  importance. 
By  combining  all  the  resulting  rankings,  it  becomes  clear  which  of  the 
factors  are  most  important.  Exhibit  III- 1  shows  the  results  of  such  an 
exercise  carried  out  in  the  U.K.  market  in  1985. 

This  approach  can  be  supported  by  individual  product  analysis,  as 
illustrated  in  Exhibit  II1-2.  This  shows,  for  example,  that  for  large 
systems,  98.9%  of  users  regard  response  time  as  an  important  factor, 
but  only  8.6%  of  users  regard  it  as  the  most  important  factor. 

An  alternative  to  ranking  the  various  elements  is  to  ask  users  to  rate 
them  on  a  scale  of  importance.  The  net  result  may  often  be  the  same, 
but  there  does  seem  to  be  a  temptation  for  users  to  rate  every  element 
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EXHIBIT  lll-l 


RELATIVE  IMPORTANCE  OF  SERVICE  FACTORS 
(Average  Ranking) 


System  Availability 


Reliability 


Response  Time 


Repair  Time 


Price 


Software  Maintenance 


Up-Time  Guarantees 


Preventive  Maintenance 


Engineer  Continuity 


Remote  Diagnostics 


Choice  of  Service 


6.  3 


J 


^3 


5.4 


5.2 


I 


4.4 


3.6 


3.  3 


9.6 


J 


9.6 


J 


8.6 


8.0 


] 


Low 


Number  of  Respondents:  210 


Average 


High 


Average  Ranking  of  Importance 
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EXHIBIT  III-2 


RESPONSE  TIME  -  USER  RANKING  OF  IMPORTANCE 


PERCENT  OF  USERS  RANKING 
RESPONSE  TIME  AS: 

TOTAL  PER- 
CENT OF  USERS 

Most 
Important 
Factor 

Important 
Factor 

Neutral 

Relatively 
Unimportant 

Least 
Important 

Important 

Unimportant 

Q  CO 
O  .  D  ^ 

90.  i% 

1.1% 

98.9% 

1.1% 

Small  Sv^tpm^ 

8  8 

oo .  o 

/  .  D-Q 

1  Q 

95.  6 

1 .  9 

Peripherals  and  Terminals 

9  0 

O  J  .  J 

L  .  / 

U.  7 

92.  3 

3.  4 

Data  Communications 

6  2 

88  U 

■3  Q 

J.  y 

94.  6 

3.  9 

Personal  Computers 

3.6 

85.  0 

6.  4 

4 

n  7 

00.  0 

D.  0 

Word  Processors 

8.6 

87.  1 

2.9 

1.4 

95.  7 

1.4 

Work  Stations 

10.0 

75.0 

10.0 

5.0 

85.0 

5.0 

Copiers 

8.4 

81.4 

1.7 

8.5 

89.8 

8.5 

PBX 

16.  6 

77.  8 

5.6 

94.4 

5.6 

Total  Hardware 

8.0% 

85.  2% 

3 .  3% 

3.  1% 

0.4% 

93.2% 

3.5^ 

Number  of  Respondents:  210 
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as  being  vitally  important.  The  main  advantage  of  rating  is  that  it 
allows  comparison  of  performance  against  user  requirements,  as  shown 
in  Exhibit  III-3.  This  clearly  identifies  those  areas  requiring  manage- 
ment attention.  Another  advantage  is  that  areas  of  over-performance, 
as  well  as  problem  areas,  are  identified. 

B.       USER  COMPLAINTS 

9  As  well  as  the  'formal'  rating  or  ranking  of  the  importance  of  service  factors, 
it  is  also  useful  to  include  free-format  questions  about  service.  The  answers 
to  these  questions  may  help  to  identify  particular  areas  of  user  concern  and  to 
explain  the  ratings  given  to  some  of  the  factors.  Quite  often,  apparently 
minor  complaints  can  loom  large  in  the  user's  mind. 

®  Exhibit  III-4  shows  a  typical  analysis  of  complaints  grouped  into  a  number  of 
categories,  while  Exhibit  III-5  gives  some  typical  user  quotes  showing  the 
range  of  their  complaints. 

•  Addressing  these  complaints  may  not  in  itself  improve  some  of  the  key 
measurable  performance  areas,  but  can  nonetheless  go  a  long  way  to 
improving  user/vendor  relationships. 
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EXHIBIT  III-3 


CDC  HARDWARE  SUPPORT 
USER  REQUIREMENTS  VERSUS  LEVEL  OF  SERVICE  RECEIVED 


10 


Received  >  Required 


Required  >  Received 


5  6  7 

Service  Required* 


10 


A. 
B. 
C. 
D. 
E. 


Hardware  Capacity  Planning 
Hardware  Consulting 
Hardware  Documentation 
Hardware  Training 
Parts  Availability 


♦Rating:  1  =  Low,  10  =  High 


F.  Dispatching 

G.  Escalation 

H.  Hardware  Maintenance 

I.  Field  Engineer  Skill  Level 

Source:  INPUT  U.S.  Study 
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EXHIBIT 


WORST  FEATURES  OF  SERVICE  -  USER  VIEWS 


Source:  INPUT  Survey 
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EXHIBIT  111-5 


WORST  FEATURES  OF  SERVICE  -  USER  QUOTES 


"Increasing  cost  for  software  service  of  declining  quality,"  ICL  User 


"High  costs  for  limited  expertise,"  DEC  User 

"Inexperienced  engineers,  lack  of  hardware  and  software  diagnostic 
tools,  unreliability  of  equipment  and  software,"  ICL  User 


"Lack  of  training  given  to  customer  service  engineers  on  new 
products,"  ICL  User 


"Lack  of  expertise  on  both  hardware  and  software  from  suppliers," 
IBM  and  Honeywell  User 


"PCB  swapping  between  units  to  determine  fault, "  ICL  User 

"Failure  to  achieve    first-time  fix,  particularly  on  peripherals  such 
as  printers,"  Systime  User 

"Inadequate  diagnostic  skills  -  fault  recurring  within  24  hours,"  ICL 
User,  serviced  by  Mills  Associates 

"Engineers  often  reduced  to  'Let's  try  this  and  see  what  happens'," 
DEC  User 


"Different  groups  in  the  supplier  organization  do  not  communicate; 
my  problems  get  passed  around,"  Hewlett-Packard  User 


"Long  call-to-fix  time  does  not  recognise  the  importance  of  systems 
to  our  business  function,"  ICL  User 


-41  - 

©1985  by  INPUT.  Reproduction  Prohibited. 


INPUT 

FBMS 


r 


IV       ANALYSIS  OF  RESULTS 


A.       TARGETING  THE  PROBLEM 


•  Customer  satisfaction  surveys  should  result  in  an  action  plan  aimed  at 
improving  customer  satisfaction  with  the  service  being  provided.  The  survey 
must,  therefore,  aim  at  those  areas  which  are  of  most  concern  to  the  user- 
reliability,  availability,  and  response  time, 

•  Each  of  these  areas  is,  of  course,  dependent  on  a  range  of  sub-factors  and  are 
in  themselves  interrelated.  Effective  targeting  of  the  problem  areas  depends, 
therefore,  on  the  isolation  and  measurement  of  these  sub-areas, 

•  Reliability  can  be  looked  at  as  inherent  machine  reliability,  but  over  a  period 
of  time,  the  effectiveness  of  any  fault  repair  or  preventive  maintenance  will 
have  an  impact  on  the  user  view  of  system  reliability,  A  study  into  this  area 
should,  therefore,  not  only  attempt  to  measure  inherent  machine  reliability, 
but  also  the  effectiveness  of  maintenance  and  repair  activity  on  that  equip- 
ment. In  this  way,  it  will  be  possible  to  define  the  problem—be  it  with  design, 
manufacturing,  or  service  and  support— which  has  to  be  solved, 

•  Availability  also  depends  on  a  number  of  factors,  some  system  oriented,  some 
vendor  oriented,  and  some  user  oriented.  One  important  fact  to  establish  is 
the  user's  pattern  of  system  usage,  as  this  is  likely  to  determine  both  the 
potential  failure  rate  of  the  system  and  the  criticalness  of  the  availability  to 
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the  user.  There  is  a  close  affinity  between  reliability  and  system  availability, 
but  there  are  also  a  number  of  areas  under  the  service  vendor's  control  which 
can  impact  availability: 

Response  time. 

Repair  turnaround  time. 

Timing,  frequency,  and  duration  of  preventive  maintenance  activities. 

•  If  availability  is  seen  by  the  user  to  be  a  problem,  then  it  is  important  to 
identify  precisely  where  the  problem  lies  and  what  is  the  root  cause  of  the 
poor  level  of  availability  so  that  the  appropriate  action  can  be  taken. 

•  Response  time  is  one  service  activity  which  users  frequently  complain  about. 
In  targeting  a  problem  in  response  time,  it  is  important  first  to  ensure  that 
both  user  and  vendor  are  measuring  the  event  in  the  same  way.  From  that 
point,  check  through  the  response  chain  to  identify  areas  of  potential  bottle- 
neck within  the  dispatching  system.  Often,  however,  the  biggest  problem  lies 
in  the  lack  of  availability  of  suitably  qualified  engineers— a  point  again  often 
made  by  users. 

B,       CONCENTRATING  RESOURCES 

•  The  greatest  benefit  from  effective  problem  targeting  is  the  ability  to 
concentrate  all  necessary  corporate  resources  on  solving  the  real  problem  and 
not  wasting  energy— and  money— on  solving  apparent  problems  instead  of  real 
ones. 

•  Once  the  problem  has  been  clearly  defined,  the  necessary  action  can  be 
initiated— action  which  can  take  many  forms: 
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Recruit  more  engineers. 
Improve  the  training  of  engineers. 

Optimise  the  geographic  distribution  of  engineers  in  relation  to  sites 
being  supported. 

Optimise/improve  parts  holding  capability. 
Improve  or  establish  remote  diagnostic  systems. 

•  These  are  obviously  only  examples  of  what  could  be  a  very  long  list  of  poten- 
tial action  points  emerging  from  a  customer  satisfaction  survey. 

•  As  with  any  action  plan,  it  is  important  not  only  to  detail  the  actions  to  be 
taken,  but  also  to  identify  who  will  be  responsible  for  the  individual  action 
points,  the  time  frame  in  which  the  action  will  take  place,  and  when  the 
desired  results  should  be  achieved. 

•  By  following  this  basic  approach,  management  can  optimise  the  use  of 
increasingly  scarce  and  expensive  resources. 

C.       DEVELOPING  THE  SOLUTION 

•  This  follows  naturally  from  the  careful  identification  of  the  problem  described 
above.  The  added  dimension  when  developing  the  solution  is  the  introduction 
of  the  targeted  result. 

In  this  process,  it  is  important  to  identify  the  desired  result.  The 
problem  may  be  that  the  response  does  not  meet  the  users'  ideal 
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requirement,  but  the  solution  may  well  be  to  improve  current  perform- 
ance, not  necessarily  to  meet  that  user  ideal. 

As  can  be  seen  from  many  of  the  Exhibits  in  Chapter  11,  the  user  satisfaction 
charts  do  not  follow  a  linear  path,  but  rather  grow  by  steps.  The  implication 
for  a  service  vendor  is  that  a  small  improvement  in  system  availability  or 
response  time  will  not  necessarily  result  in  an  increase  in  the  number  of 
satisfied  users.  it  may  be  necessary  to  improve  performance  from,  for 
example,  a  90%  availability  right  up  to  95%  in  order  to  improve  user  satis- 
faction significantly. 

Also  at  this  point,  management  must  decide  whether  the  problem  is  real  or 
only  perceived.  Again,  as  has  been  shown  earlier  in  the  report,  there  is  often 
a  disagreement  between  the  vendors'  statistics  and  user  experience—or 
perceived  experience.  The  solution  to  this  perceived  problem  may,  therefore, 
be  a  communication  solution  rather  than  an  'actual'  solution. 

The  path  to  increased  customer  satisfaction  through  measurement  is 
summarised  in  Exhibit  iV-l. 
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EXHIBIT  IV-1 

BENEFITS  OF  MEASURING  CUSTOMER  SATISFACTION 


Identifying  Real  Problem  Areas 
To  Facilitate 
Concentrating  Resources  on  the  Problem 
Leading  to 
Solving  the  Problem 
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